Skip to content

feat(cli): fast and no more JIT openapi spec - #13

Open
nimarb wants to merge 6 commits into
mainfrom
nimar/no-more-specli
Open

feat(cli): fast and no more JIT openapi spec#13
nimarb wants to merge 6 commits into
mainfrom
nimar/no-more-specli

Conversation

@nimarb

@nimarb nimarb commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@nimarb
nimarb requested a review from a team as a code owner July 17, 2026 15:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac7e6a7566

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json
Comment thread src/cli.ts
Comment thread src/cli.ts
Comment thread package.json
Comment thread src/cli.ts Outdated
Comment thread package.json
Comment thread src/cli.ts Outdated
@nimarb

nimarb commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previously reported blocking issues (missing tsconfig for typecheck, greedy boolean-flag parsing, and the --output null-body bug) all appear fixed in this revision. This round's findings are nits only (duplicated location-mapping/kebab-case logic, and a narrow nested-boolean-flag edge case with an existing --body-json workaround). Beyond those, I checked the codex-reported __schema --json legacy-envelope regression and the duplicated LEGACY_FIELD_FLAGS_UNSUPPORTED set in the test file — both are fine as-is (the README now documents the new discovery contract, and the test's copy is intentional isolation from the runtime source, not accidental drift).

Extended reasoning...

This PR is a large, complex rewrite of the CLI's request pipeline, build process, and CI/release scripts (26 files), including auth-header construction in src/client.ts and a new OpenAPI-contract compiler. Given that scope and the fact that a prior review round surfaced real P1/P2 bugs (now fixed), I don't think this warrants approval outright, but this round's findings are limited to nits with no new blocking issues, so I'm leaving this as a short status note rather than a full deferral write-up.

Comment thread src/cli.ts
Comment thread src/contracts/compiler.ts Outdated
Comment thread src/cli.ts
@nimarb

nimarb commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

@claude review

Comment thread src/contracts/compiler.ts
Comment on lines +1 to +2
import { parse } from "yaml";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The command-naming disambiguator in planCommandNames (conformance/src/naming.ts, wired into the runtime CLI by this PR) produces stuttered/opaque action names for three new 4.10.0 dashboard-placement operations: unstable_dashboards_deletePlacement -> delete-delete-placement, unstable_dashboards_updatePlacement -> update-update-placement, and unstable_dashboards_addPlacement (POST with a path arg) -> the uninformative post. Users must type e.g. langfuse api unstable-dashboards delete-delete-placement <dashboardId> <placementId> — the commands work correctly, this is purely a naming-quality issue.

Extended reasoning...

What the bug is. planCommandNames in conformance/src/naming.ts infers a CLI action verb from HTTP method + operationId suffix, then runs a collision disambiguator when two operations on the same resource map to the same action. For the three new placement operations added by this PR's 4.10.0 snapshot, the generated action names are low quality:

  • unstable_dashboards_deletePlacement (DELETE /dashboards/{dashboardId}/placements/{placementId}) collides with unstable_dashboards_delete on unstable-dashboards:delete. The disambiguator strips the resource prefix from unstable-dashboards-delete-placement, leaving delete-placement, then re-prepends the inferred action verb (delete) — producing the stuttered delete-delete-placement.
  • unstable_dashboards_updatePlacement follows the identical path with PATCH -> update, producing update-update-placement.
  • unstable_dashboards_addPlacement (POST /dashboards/{dashboardId}/placements) has a path argument, so the POST && !hasPathArg -> create heuristic doesn't fire. The operationId suffix dashboards_addPlacement isn't a recognized canonical verb either, so inferAction falls all the way through to kebabCase(method), yielding the bare HTTP verb post as the action name. It's unique within unstable-dashboards, so no disambiguation runs and post stands as the final name.

Why nothing prevents this today. This logic is new/newly-exercised by this PR (it previously only backed conformance test generation; this PR wires planCommandNames into src/contracts/compiler.ts to drive the actual runtime CLI command surface). The disambiguator's prefix-stripping only checks whether the resource prefix or an action-synonym prefix matches — it never checks whether the already-inferred action verb also appears at the start of the remainder after stripping, so when the operationId suffix itself begins with the action word (deletePlacement, updatePlacement), the verb gets attached twice. Similarly, inferAction's POST-to-create shortcut is conditioned on having no path argument, which is a reasonable general heuristic but leaves any POST-with-path-arg operation whose suffix isn't independently recognized to fall through to the raw HTTP method.

Impact. All three commands are fully functional — I confirmed with the verifiers that the conformance suite drives updatePlacement/deletePlacement successfully via the compiled schema, and each name is unique and routes to the correct operation. The only effect is that users typing langfuse api unstable-dashboards <action> ... see confusing/uninformative action names (delete-delete-placement, update-update-placement, post) in --help output and have to use those exact strings on the command line.

Step-by-step proof (matches independent empirical verification against the compiled 4.10.0 contract by three verifiers):

  1. unstable_dashboards_deletePlacement: resource = unstable-dashboards. inferAction sees DELETE + a path arg present -> action = delete. This collides with unstable_dashboards_delete's action (also delete) on the same resource, so the disambiguator runs on the full slug unstable-dashboards-delete-placement. It strips the resource prefix unstable-dashboards-, leaving delete-placement. It then returns ${action}-${name} = delete-delete-placement.
  2. unstable_dashboards_updatePlacement: identical shape, PATCH -> action update, collision with unstable_dashboards_update -> final name update-update-placement.
  3. unstable_dashboards_addPlacement: POST with path arg dashboardId present, so the create-shortcut doesn't apply; suffix dashboards_addPlacement doesn't match any canonical verb; falls through to kebabCase('POST') = post; unique within unstable-dashboards, so it's the final action name with no disambiguation needed.

Suggested fix. In the disambiguator, after stripping the resource/synonym prefix, check whether the remainder already starts with the inferred action verb (or a recognized synonym of it) before re-prepending — e.g. only prepend when the remainder doesn't already begin with that token, so delete-placement stays delete-placement instead of becoming delete-delete-placement. For addPlacement, consider extending the verb-recognition list (or the operationId-suffix parser) to map add -> create/add independent of whether a path argument is present, since "add a sub-resource to a parent identified by path" is a common and reasonable pattern that the current no-path-arg restriction doesn't anticipate.

One verifier argued this shouldn't be filed since the outputs are technically valid, unique, and deterministic. I agree the commands are not broken — that's why this is a nit, not a blocking issue — but the double-verb stutter and bare-HTTP-verb fallback are clearly unintended artifacts of the prefix-stripping logic rather than deliberate design choices, and they will confront every user of these three new endpoints, so it's worth a follow-up fix.

Comment thread src/cli.ts
Comment on lines +540 to +557
if (parameter.required && target[parameter.name] === undefined) {
throw new CliError(`Missing required option --${parameter.cliName}`);
}
}
let body = completeBody ?? fieldBody;
if (completeBody === undefined && operation.requestBody?.legacyFieldFlags) {
const missing = operation.requestBody.fields
.filter((field) => field.required && fieldBody?.[field.name] === undefined)
.map((field) => `--${field.name}`);
if (missing.length > 0) {
throw new CliError(`Missing required body option(s): ${missing.join(", ")}`);
}
if (body === undefined && operation.requestBody.required) body = {};
}
if (operation.requestBody?.required && body === undefined) {
throw new CliError(`${operation.operationId} requires a request body`);
}
if (body !== undefined) input.body = body;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 In parseOperationInput (src/cli.ts), let body = completeBody ?? fieldBody; uses ??, which treats an explicit JSON null from --body-json null/--body-file the same as undefined and silently replaces it with fieldBody. This causes a misleading "requires a request body" error for required-body operations, or silently drops the null for optional ones — fix by using completeBody !== undefined ? completeBody : fieldBody, matching the strict === undefined check the adjacent guard already uses.

Extended reasoning...

parseOperationInput in src/cli.ts builds completeBody from --body-json/--body-file, which can legitimately be the JSON value null (e.g. a user running --body-json null). At the line let body = completeBody ?? fieldBody;, the ?? operator treats null the same as undefined, so an explicit null completeBody gets silently replaced by fieldBody (which is undefined when no legacy field flags were passed).

This produces two distinct, incorrect behaviors depending on whether the operation's request body is required:

  1. Required body: the subsequent check operation.requestBody?.required && body === undefined throws "<operationId> requires a request body" — even though the user explicitly supplied one via --body-json null. The error message is actively misleading in this case.
  2. Optional body: since body !== undefined is false, input.body is never set on the call input, so no body is sent at all — silently changing the request from "send an explicit null body" to "send no body," which is a different wire behavior.

Notably, a few lines above this in the same function, the guard that decides whether to fall back to legacy field flags uses a strict completeBody === undefined check — correctly distinguishing null from undefined. This shows the author's intent was to treat an explicit null as a real value, but the very next use of completeBody (the ?? on the body-resolution line) fails to apply that same distinction. It's an internal inconsistency within the same function, not just a stylistic nit.

Step-by-step proof: For an operation with requestBody.required = true, calling parseOperationInput with tokens ["<id>", "--body-json", "null"]:

  • --body-json null is parsed via JSON.parse("null"), so completeBody = null.
  • The strict guard completeBody === undefined && ... evaluates to false (correctly, since null !== undefined), so the legacy-field-flag branch is skipped as intended.
  • At let body = completeBody ?? fieldBody, since ?? treats null as nullish, body becomes fieldBody, which is undefined.
  • The required-body check then throws "widgets_update requires a request body", contradicting the fact that the user did supply a body.
  • For the optional-body case, body !== undefined is false, so input.body is never set, and the explicit null is dropped entirely.

Why nothing else catches this: only literal JSON null triggers the bug — falsy-but-non-nullish values like false, 0, or "" are not affected by ?? and pass through correctly, so this only surfaces for the specific edge case of an explicit top-level null body.

Fix: replace the ?? with a strict check, matching the pattern already used a few lines above: let body = completeBody !== undefined ? completeBody : fieldBody;

Severity note: all three independent verifiers who examined this confirmed the bug is real and reproducible, but rated it nit because no current Langfuse API operation actually accepts a bare JSON null as its top-level request body — every documented request body is an object (e.g. SubmitFeedbackRequest, CreateScoreRequest). So while the code is genuinely inconsistent with its own adjacent guard and would confuse a user who explicitly tries --body-json null, it doesn't affect any realistic invocation of the CLI today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant